As manufacturing capabilities have improved for image sensor devices, it has become possible to place more pixels in a fixed-size area of silicon. As a consequence, pixel size is shrinking. From a signal processing perspective, more pixels imply that the scene is sampled at a higher rate providing a higher spatial resolution. Smaller pixels, however, collect less light (photons) which, in turn, leads to smaller per-pixel signal-to-noise ratios (SNRs). This means as light levels decrease, the SNR in a smaller pixel camera decreases at a faster rate than SNR in a larger pixel camera. Thus, the extra resolution provided by a smaller pixel image sensor comes at the expense of increased noise.
A side effect of placing more pixels into a fixed-sized silicon sensor is lower pixel well capacity. As pointed out earlier, less photons result in reduced signal everywhere. The impact of reduced signal is particularly severe in blue regions of the image such as the sky. Because each pixel element receives fewer photons, the red channel signal in blue regions is particularly weak (due to the use of Bayer color filter arrays) which, after amplification from white balancing, color correction and local tone mapping manifests itself as noise in blue regions of the image. One approach to this problem would be to enhance the noise reduction strength for blue pixels. This will mitigate noise in blue regions such as sky, but would also result in the removal of texture in other blue regions such as ripples in water, ocean waves, and blue jeans or shirts. Another approach would be to extract regions of the image that contain large relatively smooth blue regions (e.g., sky) using image segmentation techniques and a learning-based method to separate these types of regions from the rest of the image. Noise reduction strengths could then be enhanced in these regions. Image segmentation is, however, a very time consuming and processor-intensive process and is not feasibly implemented in a camera pipeline.
Sharpness and noise are arguably the two most important image quality considerations for an image. Camera manufacturers would like to deliver an image that is sharp with very low noise. Since edges/texture and noise overlap in frequency, often times these are conflicting goals. Typically noise reduction results in a softer image while classical sharpening methods enhance high frequency content in the image (both signal and noise). The challenge is to devise a methodology that removes noise in smooth areas where it is most visible while enhancing sharpness in texture-rich regions.